MCAR: The Happy Case

• Missingness is truly random.
• For MCAR, we randomly delete nutrition (X) for a subset of mothers.
• Random missingness has no relationship to education or nutrition values.

MCAR: Impact

• Missing data does not cause biased estimates. It just reduces precision/power.
• Example: random equipment failures, accidentally dropped samples.

MCAR: Dag

Simulation Code: Base Data and MCAR

# Ensure base data exists (5000 rows defined earlier)
if (!exists("sim_data")) {
  sim_data <- simulate_birth_data(
    n         = 5000,
    beta_x    = beta_x_true,
    beta_e    = beta_e_true,
    sd_error  = sd_error_true
  )
}

# Full data model (gold standard) on the complete data
full_fit <- lm(Y ~ X + E, data = sim_data)

# Randomly mark 50% of X as missing (MCAR) for this demo
mcar_data <- sim_data |>
  mutate(
    drop_flag = rbinom(n(), 1, 0.5),
    X_mcar = if_else(drop_flag == 1, NA_real_, X)
  )

MCAR with Listwise Deletion

# Only keep rows where X_mcar is observed (listwise deletion)
mcar_cc <- mcar_data |> filter(!is.na(X_mcar))
mcar_fit <- lm(Y ~ X_mcar + E, data = mcar_cc)

nrow(mcar_cc)

[1] 2522

summary(mcar_fit)

Call:
lm(formula = Y ~ X_mcar + E, data = mcar_cc)

Residuals:
    Min      1Q  Median      3Q     Max 
-7.7022 -1.3534  0.0086  1.3901  7.7041 

Coefficients:
            Estimate Std. Error t value Pr(>|t|)    
(Intercept)  1.97448    0.05698   34.65   <2e-16 ***
X_mcar       0.47749    0.04102   11.64   <2e-16 ***
E            1.02528    0.10239   10.01   <2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 2.043 on 2519 degrees of freedom
Multiple R-squared:  0.1916,    Adjusted R-squared:  0.1909 
F-statistic: 298.5 on 2 and 2519 DF,  p-value: < 2.2e-16

Summary of Differences

Call:
lm(formula = Y ~ X_mcar + E, data = mcar_cc)

Coefficients:
(Intercept)       X_mcar            E  
     1.9745       0.4775       1.0253  

# A tibble: 2 × 3
  method            estimate     se
  <chr>                <dbl>  <dbl>
1 Full data (truth)    0.500 0.0288
2 MCAR listwise        0.477 0.0410

Power Analysis: MCAR vs. Complete Data

MCAR: estimates center on truth; uncertainty widens as more X is missing.

MCAR: Not a Big Deal But Rarely Plausible

• MCAR is the happy case: just drop mothers with missing nutrition, move on.
• You lose precision/power, but no bias.
• Problem: MCAR is seldom plausible, except in a few cases (e.g., random measurement instrument failure)

MAR: Nonresponse Driven by E and X

• Missingness rises with an observed factor (e.g., education, clinic context) and when nutrition is low.
• Education affects both nutrition and birthweight.
• Dropping cases with missing values up-weights higher-education/higher-nutrition mothers, biasing the nutrition effect.

MAR Scenario Set

We will look at:

• A single “truth” dataset (n = 20,000) with no missingness
• MAR cases with 10%, 30%, and 60% of X missing
• For each level: one scenario where missingness is driven by E (observable) and one where missingness is driven by X itself (worst case)

DAG

MAR Scenarios, summarized

• No Y/E bias here; we only are missing on X here
• Bias increases with more missing data
• Bias increased when relationship between X, E, and p(missing) is higher
• The highest bias scenario has a particularly strong link between the true value of X and p(x missing)

MAR Scenarios: Naive Regression (Y ~ X)

• No adjustment for E; drop rows with missing X.
• True \(β_X\) = 0.50
• E affects both Y, X, and p(x missing); these models are both MNAR and mis-specified
• MNAR and mis-specified models tend to go hand-in-hand.

MAR Scenarios: Adjusted Regression (Y ~ X + E)

• This model is correctly specified. “No backdoor paths.”
• Observed scenarios drop cases with missing X.
• True \(β_X\) = 0.50
• Even without any imputation, the adjusted model recovers \(β_X\) reasonably well across scenarios.
• The main consequence is power loss, not bias.

When MAR Holds (and When It Doesn’t)

• If we model the variables that drive missingness (here, X and E), listwise deletion analyses stay near the truth.
• In that case, missing data mostly hurts precision: fewer cases translates to bigger SEs.
• Trouble begins when key drivers are unmeasured or omitted (e.g., X-driven missingness with only Y ~ X): then MAR is violated relative to the analysis, and bias appears.
• In real workflows we can combine modeling plus imputation; this performs best.

Common, Non-Ideal Imputation Tactics

• We apply four historically common tactics to each MAR scenario (E-driven and X-driven at 10/30/60% missing) plus the full, no-missing baseline.
• Every regression keeps \(E\) in the model (since we observe it); what changes is how we reconstruct \(X\).
• Tables on the next slides show \(β_X\), standard errors, and how far each method drifts from the true 0.50.

Mean Imputation

• Insert a single cohort-wide mean for every missing nutrition value.
• Mechanics: compute the mean of observed \(X\), then plug that constant into each missing slot
• All imputed mothers receive the same nutrition score, so their points line up on the same vertical slice of the scatter.

Mean Imputation: Simulation Results

Mean Imputation vs. Listwise Deletion

Mean Imputation: Problems

• Even with \(E\) in the regression, bias is worse with mean imputation than just using listwise deletion.
• In this case, shrinks the slope toward zero: missing cases on average have worse nutrition; we replace those missing values with the data from cases where highly educated, well-nourished mothers are over-represented.
• Standard errors are too small because we flooded our data with identical, non-varying values.

Mean + Indicator

• Start with mean imputation, then append a binary “missing” indicator to the regression.
• Mechanics: fill missing \(X\) with the cohort mean, create miss_ind = 1 when \(X\) was imputed, and fit $Y X + E + miss_ind`.
• Intuition: mothers with missing nutrition get the same slope as everyone else but a separate intercept shift governed by the indicator.

Mean + Indicator: Simulation Results

Mean + Indicator vs. Listwise Deletion

Mean + Indicator: Problems

• In this case, adding an indicator reduced bias a good deal compared to mean imputation
• Treats missingness as a fixed effect, so can exacerbate bias on other coefficients (e.g., \(\beta_E\)) correlated with missingness

Hot-Deck Imputation: Mechanics

• For each missing nutrition value, randomly draw a donor mother who reported \(X\) and copy her score.
• Mechanics: sample with replacement from observed \(X\)’s, replace the NA’s, and analyze \(Y \sim X + E\) on the filled in dataset.

Hot-Deck Imputation: Simulation Results

Hot-Deck Imputation vs. Listwise Deletion

Hot-Deck Imputation: Problems

• In this case, performs worse than mean imputation, vastly exacerbating bias from listwise deletion
• Works when missingness is MCAR; once it depends on \(E\) or \(X\), donor pools misrepresent the missing cases.
• Still leads to overly tight SEs because draws are constrained to known cases. In our case, these known cases are from a tighter distribution than the truth.

Regression Imputation: Mechanics (Single Driver)

• Estimate a predictive model \(\widehat{X} = \widehat{\alpha} + \widehat{\gamma} E\) using only mothers with observed nutrition.
• \(\widehat{\alpha}\) is the fitted intercept (baseline nutrition when \(E = 0\)); \(\widehat{\gamma}\) is the estimated effect of \(E\) on nutrition.
• Mechanics: use the observed \(E\) to predict \(X\) for every missing case, replace the NA with that prediction, and then run the outcome regression.
• In our simulation, low-education mothers receive imputations near \(\widehat{\alpha}\) while high-education mothers get \(\widehat{\alpha} + \widehat{\gamma}\)—mirroring the DAG structure instead of collapsing to a single mean.

Regression Imputation: Mechanics (Multiple Drivers)

• When several observed variables \(Z_1, Z_2, \ldots, Z_p\) relate to nutrition or missingness, fit \(\widehat{X} = \widehat{\alpha} + \sum_{j=1}^p \widehat{\gamma}_j Z_j\) among mothers who reported \(X\).
• Each coefficient \(\widehat{\gamma}_j\) shows how that covariate shifts expected nutrition; the imputed value is the linear predictor for each missing case.
• In richer data, the \(Z_j\) could include clinic context, SES, prior visits, etc.—anything observed that plausibly drives both nutrition and missingness.

Model-based Imputation Techniques

• With rich data sets, you do not have to limit your missing data model to covariates in your analysis model
• The idea is to model each variable with the most informative model you have
• This is similar to propensity score models, where you model the treatment mechanism to produce matched sets or weights to achieve balance

Regression Imputation: Simulation Results

Regression Imputation vs. Listwise Deletion

Regression Imputation: Problems

• SEs are typically too optimistic: every missing X is deterministically set
• Multiple imputation could add back the right amount of noise and uncertainty.
• Remember: the goal is unbiased estimates of both \(\beta_X\) and \(SE(\beta_X)\)

Deterministic Imputation ≠ Power

• Same line, more dots (Enders): Regression imputation replaces each missing \(X\) with its predicted value from \(E\), so the imputed points fall exactly on the fitted line. With correlation between the imputed \(X\) and \(E\) essentially one, those rows reinforce the complete-case relationship instead of adding new leverage.
• Spread, not headcount, drives SEs: The standard error of a slope depends on the spread of the predictor, not the raw row count. Because all imputed \(X\)’s sit on that line, they add almost no variance, so the denominator of \(\mathrm{SE}(\hat\beta_X)\) barely changes. Worse, single-value imputation pretends those fills are certain, so standard errors can be too small (Gelman & Hill).
• Where false confidence lives: The problem isn’t complete-case analysis—it’s single imputation that ignores the extra uncertainty. Complete cases lose power but report honest SEs; deterministically imputed cases keep \(N\) high but understate uncertainty and still inherit bias when missingness isn’t MCAR.

Where Do We Go From Here?

• We need to put believable spread back into the missing rows rather than printing the same predicted value over and over.
• Multiple imputation helps mitigate this problem

Multiple Imputation

• Treat the missing nutrition scores as random draws from a model that predicts \(X\) using the information we do have (here, \(E\)).
• Do that filling process multiple times, fit \(Y \sim X + E\) in each completed data set, and then average the estimates using Rubin’s rules.

Multiple Imputation: Simulation Results

Multiple Imputation vs. Listwise Deletion

Why Multiple Imputation Helps

• Adds plausible wiggle: MI draws \(X^* = f(E) + \varepsilon\), where \(f(E)\) is the best guess for nutrition given \(E\) and \(\varepsilon\) is a random residual sampled from the estimated spread. That restores the variation we would have seen if \(X\) were observed.
• Honest uncertainty: Rubin’s imputation combination rules blend the within- and between-imputation variance, so SEs stay truthful even when missingness is heavy.
• Bias correction: Because each draw reflects how \(X\) and \(E\) relate, the averaged estimate lands near the true \(\beta_X\).

Multiple Imputation: Takeaways

• MI lands on the true \(\beta_X\) even in the harsh scenario where values of X drive missingness.
• The combined SEs stay honest because Rubin’s rules keep the run-to-run variation in the mix.
• Next session we’ll dig into more details on how MI works and how to implement it yourself.

MNAR & Imputation

• Multiple imputation is not a magic bullet
• If you do not have rich enough covariates to accurately model p(missing X), there is no missing data strategy that can save your analysis.
• Lets finish by examining such a scenario

MNAR DAG: Hidden Drivers

MNAR Stress Test Setup

• Nutrition, income, provider quality, family stability, and class are all correlated.
• We only observe education and insurance, so any missingness tied to the other pieces is effectively MNAR for us.
• Lets compare listwise deletion vs. MI when we have a complex causal graph with unobserved data

MNAR Simulation Details

• Simulated 2,000 mothers with seven covariates; truth: \(\beta_X = 0.50\) in \(Y \sim X + \text{education} + \text{insurance}\).
• Missing nutrition is more likely for low income, low provider quality, fragile family, lower class position mothers.
• Analysts only get \(Y\), education, insurance, and nutrition (with missingness).

MNAR Table 1: Who’s in Each Sample?

• We can see here by using MI, our summary statistics look more accurate
• The MAR illustration showed how listwise deletion recovered accurate model estimates
• However, an accurate Table 1 of summary statistics also benefits from imputation!

MNAR Model Specs

• Truth (omniscient): Y ~ X + education + insurance + income + provider + family + class.
• What we can actually fit: \(Y \sim X + \text{education} + \text{insurance}\) on whatever \(X\) values we have.
• We compare three versions of that observed model: full data (for reference), listwise deletion, and MI.

MNAR Results: Listwise vs. MI

• Omniscient model (with hidden drivers) nails \(\beta_X\), as expected.
• Even with full nutrition but no hidden drivers, we drift because the omitted variables matter.
• Once nutrition goes missing, both listwise and MI lean on education/insurance; MI can actually look worse than listwise deletion!

MNAR: No Easy Fix

• Both listwise deletion and MI miss the truth because neither sees the hidden drivers of missingness.
• MI can even overshoot when missingness leans on factors we never measure—its imputed \(X\) just mirrors education/insurance again.
• Only extra information (new variables, external data, or sensitivity analyses) can break out of this corner.

Takeaways

• Handling missing data well is all about good causal inference, on both Y and your missing Xs
• No missing data strategy can fix an impoverished data set
• You need to spend a good deal of time thinking about your causal graphs/DAGs before you implement imputation strategies
• Don’t be afraid of simulating your specific data; you can’t observe your missing Xs, but you can simulate scenarios that tell you what would happen to your results under various conditions.

Thank You!

• Erik Westlund
• Johns Hopkins Biostatistics Center
• ewestlund@jhu.edu